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Abstract 

This paper presents recent results from Mean Field Game theory underlying the intro- 
duction of common noise that imposes to incorporate the distribution of the agents as a 
state variable. Starting from the usual mean field games equations introduced in |1H [T2t [T3] 
and adapting them to games on graphs, we introduce a partial differential equation, often 
referred to as the Master equation (see [H]), from which the MFG equations can be deduced. 
Then, this Master equation can be reinterpreted using a global control problem inducing the 
same behaviors as in the non-cooperative initial mean field game. 

Introduction 

Mean field games have been introduced in 2006 by J.-M. Lasry and P.-L. Lions [11 ^ 1121 [T3] 
to generalize stochastic differential games to games involving a continuum of players. 
Since 2006, many applications of mean field games have been proposed, particularly in eco- 
nomics (see for instance [6], [9], or [10]) and an important effort has been made to solve the 
partial differential equations associated to mean field games when both time and the state 
space are continuous (see [1], [2], [8], [10], etc). 

In this paper, we consider a mean field game on a complex discrete state space that is rep- 
resented by a graph and we explain the discrete counterpart of the MFG equations and of 
the Master equation formally introduced in [H]. This Master equation, or in our discrete 
case the Master equations, "contains" the MFG equations and allows for the introduction 
of random noise (although this question is not tackled here). In addition to these Master 
equations that can be exposed in a simple way in this discrete context without relying on 
differential calculus in a space of probability measures, we provide an interpretation of them 
as the equations solved by the partial differentials of a function that solves a single Hamilton- 
Jacobi equation in high dimension associated to a global optimization problem on the graph. 

In the first section, we introduce the setting of our mean field game on a graph and 
define what a Nash-MFG equilibrium is. Then, in section 2, we define the MFG equations 
in the case of a graph, along with the Master equations, and we show how to deduce a 
Nash-MFG equilibrium from smooth solutions of these equations. In section 3 we define 
a global optimization problem on the graph and show how the solution of the associated 
Hamilton-Jacobi equation is related to the Master equations and eventually to the initial 
mean field game. 
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1 Mean field games in a graph 



We consider a directed graph Q whose nodes are indexed by integers from 1 to N . For each 
node i G A/" = {1, . . . , N} we introduce V{i) C J\f\{i\ the set of nodes j for which a directed 
edge exists from i to j. The cardinal of this set is denoted di and called the out-degree of 
the node i. Reciprocally, we denote V~^{i) C J\f\ {i} the set of nodes j for which a directed 
edge exists from j to i. 

This graph G is going to be the state space of our mean field game and we suppose that 
there is a continuum of anonymous and identical players of size 1, playing a game on the 
graph as described in the next paragraph^. 

Each player's position is represented by a continuous-time Markov chain {Xt)t with val- 
ues in Q and instantaneous transition probabilities at time t described by a collection of 
functions Xt{i, •) : V{i) — >• M_|_ (for each node i G A/"). 

Each player is able to decide on the values of the transition probabilities at time t and pays 
an instantaneous coslH L{i, (Ajj)jgv(j)) to set the value of A(i, j) to Xij. 

The assumptions made on the continuous functions L(i, ■) are the following. 
First, we assume asymptotic super-linearity for each i G A/": 

Vi G Af, hm = +00 

Then, we define the Hamiltonians H{i, •): 

yi e N',p eMf^' ^ H{i,p) = sup X ■ p - L{i, X) 

and we assume that for each i G Af, H{i,-) is a function with 

VH{i,p) = argmax^^j^d, X ■ p - L{i, X) 

This assumption is satisfied as soon as for each i G M, L(i, •) is a C'^ strongly convex function 
or if L{i,X) = X^jgv(j) C{i,j,Xij) with each of the functions C{i,j, •) a strictly convex 
function. 



Now, to define the mean field game, let us introduce a domain 17 C containing 
Vn = {{xi, ■ ■ ■ ,xn) G [0, 1]*^, X^i^i Xi = 1} and two functions / and g definecjl on M x Q 
such that Vz G M, f{i,-) and g{i,-) are continuous on Cl. 

Now, let us also define the set A of admissible controls. We are only considering marko- 
vian controls and we define: 

A = {{Xt{i, j))te[o,T],ieJ\f,j&v{i) deterministic |Vi G TV, Vj G V(i),t ^ Xt{i,j) G L°°{0,T) } 

Now, for a given admissible control^ A G ^ and a given function m : t G [0, T] 1— )• 
{m{t, 1), . . . , m{t, N)) G Vn we define the payoff function Jm [0,T] x J\f x A ^ R by: 



Jmit,i, A) = E 



T 



-L{Xs, A,(X„ •)) + fiXs,m{s))) ds + g (Xt, m{T)) 



^While writing this article, we learnt during a conference organized in Rome on Mean Field Games that results 
on the limit of continuous time and finite state space M-player-games as M — J> +00 had been obtained by Diogo 
Gomes, justifying our direct passage to a continuum of players 

^There is no additional difficulty when a time dependency is added. 

•^If one adds a time dependency to /, the following results can be adapted straightforwardly. 
"^We call A a control although it is an abuse of terminology since the controls consist in the values of A. 
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for (^s)<jg[t,T] ^ Markov chain on Q, starting from i at time t, with instantaneous tran- 
sition probabihties given by {)^s)se[t,T]- 

Prom this we can adapt the definition of a (symmetric) Nash-MFG equihbrium to our 
context: 

Definition 1 (Nash-MFG symmetric equilibrium) . A differentiable function m : t € [0, T] i-^ 
(m(t, 1), . . . ,m{t,N)) G Vn is said to be a Nash-MFG equilibrium, if there exists an admis- 
sible control X A such that: 

VA eA,yie J^,Jm{0,i,X) > Jm{0,i,X) 

and ^ 

yi £ M,-^m{t,i) = J2 ^t{j,i)m{t,j) - ^ Xt{i,j)m{t,i) 
jgv-i« iGV(i) 
In that case, X is called an optimal control. 

In this definition, m stands imphcitly for the distribution of the players across the dif- 
ferent nodes. 

As for all mean field games, the problem is a fixed point problem since, starting from 
a (guessed) trajectory m, the control problem in Definition 1 induces trajectories for the 
players and to obtain a Nash-MFG equilibrium these individual trajectories must generate 
the right macroscopic trajectory m for the distribution of the players in Q. 

In the next section we are going to write the MFG equations associated to our problem, 
which we call the decentralized problem as it involves all the players, and see that these 
equations can be extracted from a set of equations called the Master equations, as introduced 
by J.-M. Lasry and P.-L. Lions and presented in [14j in a different framework. 

2 MFG equations and the Master equation 

As for any control problem we can associate, when m is fixed, a Hamilton-Jacobi equation 
to the above control problem. More exactly, since there are A'^ states, we need to write 
down N Hamilton-Jacobi equations, one for each state. However, these backward equations 
need to be coupled with a system of forward transport equations that describe the resulting 
evolution of the distribution of players. 

The graph counterpart of the two partial differential equations of classical mean field 
games (see [HI [121 [E] ) are the following: 

Definition 2 (The ^-MFG equations). The Q-MFG equations consist in a system of 2N 
equations, the unknown being t € [0, T] i— > {u{t, 1), . . . , u{t, N),m{t, 1), . . . , m{t, N)). 

The first half of these equations are the Hamilton-Jacobi equations associated to the above 
problem and consist in the following system: 

Vi G J\f, j^u{t, i) + H (i, (n(t, j) - u{t, i))jev{i)) + fih rn{t, 1), . . . , m{t, N)) = 
with u{T, i) = g(i, m{t, 1), . . . , m{t, N)). 
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The second half of these equations are forward transport equations: 

^i&M, ^m{t,i)= ^ ^^^^ {{u{t,k) -u{t,j))kev{j))'m{t,j] 



- E 



dH{i, 
dpj 



{{u{t, k) - u{t, i))keV{i)) i) 



with (m(0, 1), . . . , m(0, A'')) = mP G Vn given. 

Now, we enounce a result saying that a solution of the ^-MFG equations provides a 
Nash-MFG equilibrium and an optimal control: 

Proposition 1 (The ^-MFG equations as a sufficient condition). Let mP G Vn o,nd let 
us consider a solution {u(t, 1), . . . , u{t, N),m{t, 1), . . . , m{t, N)) of the Q-MFG equations 
with (m(0,l),... ,m(0,Ar)) = m°. 

Then t i-> m{t) = {m{t,l), . . . ,m{t, N)) is a Nash-MFG equilibrium and the relations 
Xt{i-,j) = ^%p. ((""(^j^) ~ ""(^i O)fceV(i)) define an optimal control. 

Proof: 

Let us consider an admissible control A G .4. Let us then consider an initial position 
i & J\f and a Markov chain starting in i at time 0, with instantaneous transition 

probabilities given by A. 

Since the values taken by A are bounded, there is almost surely a finite number Q of moves 
in the time interval [0, T) and we can write (almost surely): 



u(t,X^!:^^ =u{0, 



E-{^.x:'')-u{t,-^x::i) 

q=l 



+u(T,x'/^^)-u{tQ,X^f') 



where tg = and the other ijS are the time of jumps. 
Then: 

u (r, x^!^ = u{o, i) + ^{t, Xt-)dt + j^u (t„ xl''") - u (t„ 

= uiO,i)+ -^it,Xt-)dt+ Yl {u{t,j)-u(t,X^:!'^)yXtiXt-,j)dt + MT 

where {Mt)t is a local martingale starting from that happens to be a martingale since u is 
bounded. 



Hence: 



E 



'g{X^/'\m{T))\ =E\g{X^'!:\m{T-)) =u{0,i) 



+E 



£\^^{t,Xt-)+ Yl (u{t,j)-u{t,X^f'))Xt{Xt-,j) 

u{0,i) 



dt 



E 



giX^''\miT)) - I L{Xt-,Xt{Xt-,-))dt 





+E 



E 



r [^{t,Xt_)-L{Xt_,\t{Xt_,-))+ Yl {u{t,j)-u{t,X^f''))\t{Xt_,j)\ dt 
By definition of u, we have that: 

5(X°'''\m(r)) + r(-L(Xt_,At(Xt_,.)))(iil < ix(0,i) - E [ f{X^f\m{t-))dt 
Jo J L-'o 

(-L(Xt, Xt{Xt, •)) + /(^°'''\ m(i))) dt + 5(^T'''^ "^(T^)) < ^(0> 

with equahty when A is taken to be the control A as described in the statement of the 
result (because of our assumption on the hamiltonians), this control being indeed admissible 
since the function u is bounded and since each H{i, •) is C^. 

Hence: 

J^(0, i, A) < -^(0, i) = Jm{0, i, A) 
and this proves the result. □ 

Existence and uniqueness of solutions to the Q-MFG equations are now tackled. We first 
start with existence and our proof is based on a Schauder fixed-point argument and a priori 
estimates to obtain compactness. Then we will present a criterion to ensure uniqueness of 

solutions. 

For the existence result, we first start with a Lemma stating that, for a fixed m, the 
Hamilton-Jacobi equations amongst the ^-MFG equations obey a comparison principle: 

Lemma 1 (Comparison principle). Let m : [0, T] — >■ Vn be a continuous function. Let 
u : t G [0, T] !->■ {u{t, 1), . . . , u{t, N)) be a function that verifies: 

yi G Af, -^u{t, i)-H (z, {u{t, j) - u{t, i))jev(i)) - f{i, m{t, 1), . . . , m{t, N)) < 
with u(T, i) < g{i, m{T, 1), . . . , m{T, N)). 

Let V : t E [0,T] 1-^ {^{t, 1), . . . , v{t, N)) be a function that verifies: 

Vi G AT, ~v{t, i)-H (i, {v{t,j) - v{t, i))j^v(i)) - f{h m{t, 1), . . . , m{t, N)) > 
with v{T,i) > g{i,m{T,l), . . . ,m{T,N)). 
Then, Vi G AA, Vi G [0, T] , v{t, i) > u{t, i) . 

Proof: 

Let us consider for a given e > 0, a point {t*,i*) G [0,r] x M such that 

u{t\i*) -v{t*,i*) - e{T -t*) = max u{t,i) - v{t,i) - e{T - t) 

(t,i)e[0,T]xjV 

Ifi* G [0,T),then ^ {u{t,i*) - v{t,i*) - e{T - t))\^^^, < 0. Also, by definition of (i*, i*), 
Vj G V{i*), u{t*,i*) - v{t*,i*) > u{t*,j) - v{t*,j) and hence, by definition of H(i*, •): 

H (r, {v{t*,j) - vit*,i*))JeVi^')) > H (i*, iuit*,j) - uit*,^*)),^v{^*)) 
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Combining these inequalities we get: 
~u{t\i*)-H (^\{u{t*J) - u{t\inhevii^)) > ~v{t\n^ 
But this is in contradiction with the hypotheses on u and v. 

Hence t* = T and niax^j j^gjOjTlxA/" ^(^) i)~v{t, i) — e{T—t) < because of the assumptions 
on u{T,i) and v{T,i). 

This being true for any e > 0, we have that max^^ j^gjo^rixA/"^!*) ~ v{t,i) < 0. □ 

This lemma allows to provide a bound to any solution u of the N Hamilton-Jacobi 
equations and this bound is then used to obtain compactness in order to apply Schauder's 
fixed point theorem. 

Proposition 2 (Existence of a solution to the Q-MFG equations). Let G Vr^ . Under the 
assumptions made in section 1, there exists a solution {u,m) of the Q-MFG equations 
such that m(0) = m^. 

Proof: 

Let m : [0, T] Vn be a continuous function. 

Let then consider the solution u : t G [0,T] h-^ {u{t,l), . . . ,u{t,N)) to the Hamilton- 
Jacobi equations: 

Vi G ju{t, i)+H {i, {u{t, j) - u{t, i))jev{i)) + fih mit, 1), . . . , m(t, N)) = 

with u{T, i) = g{i, m{T, 1), . . . , m{T, N)). 

This function u is a well defined function with the following bound coming from the 
above lemma: 



sup||m(-,z)||oo < sup||5(i, OIU + sup||/(i, OIloo + sup|ii"(i,0)| T 

ieJV ieJV \ieAf ieAf / 

Using this bound and the fact that the functions •) are assumed to be C^, we can 
define a function fh : [0, T] Vn by: 

jev-i(i) 

_ ^ ^I^hl (^(^u{t,k) -u{t,i))k^y(^i))m{t,i) 

with (m(0, 1), . . . , m(0, N)) = m° G Vn- 

^ is bounded, the bounds depending only on the functions f{i,-), g{i,-) and H{i,-), 
ieM. 

As a consequence, if we define 6 : m G C{[0,T],Vn) ^ m € C{[0,T],Vn), 6 is a contin- 
uous function (from classical ODEs theory) with Q{C{[0,T],Vn)) a relatively compact set 
(because of Ascoli's Theorem and the uniform Lipschitz property we just obtained). 
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Hence, because C{[0,T],Vn) is convex, by Schauder's fixed point theorem, there exists a 
fixed point m to 0. If we then consider u associated to m by the Hamilton- Jacobi equations 
as above, {u,m) is a solution to the ^-MFG equations. □ 

Coming now to uniqueness, we deduce from the proof of uniqueness introduced in [HI 
[121 [13]; a criterion to ensure uniqueness of solutions of the Q-MFG equations: 

Proposition 3 (Uniqueness for the solution of the i/-MFG equations). Assume that f and 
g are such that: 

N 

V(m\m2) G Vnx'Pn, '^{fii,m\, . . . , m]v)-/(i, m^, . . . , m^))(m- -m^) > =f- = 
and 

V(m\m^) E 'PN^'PN,'^{9{h'm\,.. . ,mjf)-g{i,mj, . . . ,m%)){m}-mf) > 



i=l 



N 



m} = 



Then, if {u,m) and {u^rh) are two solutions of the Q-MFG equations, we have m = fh 
and u = u. 

Proof: 

The proof of this result is a straightforward adaptation of the classical proof of unique- 
ness for continuous MFG equations. 

It consists in computing the value of I = X]il=i ^ ((^(^' ^) ~ ""(^i ~ rh{t, i))) dt 

in two different ways. 

We first know directly that 

AT 

/ = Y.{g{i, m{T, 1), . . . , m(r, N)) - g{i, m{T, 1), . . . , rh{T, N))){m{T, i) - m{T, i)) 

i=l 

Now, differentiating the product we get, after reordering the terms that: 



T N 



m(t, 1), . . . , m{t, N)) - f{i, m{t, 1), . . . , m{t, N))){m{t, i) - m{t, i))dt 



i=l 



T N 



1=1 



H{i, {u{t, k) - u{t, i))kevii)) - H{i, {u{t, k) - u{t, i))kev{i)) 



^ ((n(t,i) - u{t,i)) - {u{t,j) - u{t,j))) ' ^ {{u{t,k) - i))fcgv(i)) 



jev(i) 



dpj 



dt 



H{i, {u{t, k) - u{t, i))fcev(i)) " ^(.^^ (^(*' ^) " ^(*' ^))A:eV(i)) 



■ {iu{t,i) - u{t,i)) - {u{t,j) - u{t,j))) 

j€V{i) 



dH{i, ' 
dpj 



{{u{t,k) - u{t,i))kevii)) 



dt 



Now, using the convexity of the hamiltonian functions, we know that: 
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T N 



m{t, 1), . . . , m{t, N)) - f{i, m{t, 1), . . . , m{t, N))){m{t, i) - m{t, i))dt 



i=l 



N 

+ Y,i9{h m{T, 1), . . . , m(T, N)) - g{i, m{T, 1), . . . , m{T, N))){m{T, i) - m{T, i)) > 

i=l 

Hence, using the hypotheses on / and g, m = rh. 
Using the maximum principle stated in Lemma 1, we hence have u = u and the result is 
proved. □ 

A simple example of functions / and g satisfying the above criterion consists in taking 
for f{i,-) and g{i,-) functions of rrii only, strictly decreasing in mj. 



Now we define the graph counterpart of the Master equation proposed by J.-M. Lasry 
and P.-L. Lions and presented in |14j : 

Definition 3 (The ^-Master equations). The Q -Master equations consist in N equations, the 
unknown being {t, mi, . . . , mjy) £ [0,T] x Q i-^ {Ui{t, mi, . . . ,mN), ■ ■ ■ , UNit, mi, . . . , mjy)). 

dUi 

yi G M, -^{t,mi, . . .,mN) + H {i, {Uj{t,mi, . . . ,mN) - Ui{t,mi, . . ■ ,mN))jeVii)) 



N 



da 



1=1 



dHjj, 
dpi 



{{Uk{t,mi, . . . ,mN) -Uj{t,mi,.. ■ ,mN))keV{j)) 



-E 



J6V(Z) 



dH{l,-) 
dpj 



{{Ukit, mi,..., mN) - Ui{t, mi,..., mN))kevii)) 



+f{i,mi, . . . ,mAr) = 



with Ui{T, mi, . . . ,mj\f) = g{i, mi, . . . , m^). 



This equation is the Hamilton-Jacobi equation associated to the decentralized problem 
when we consider that the state variable is not only the current position but also the repar- 
tition of the players on ^. In particular, when there is a common noise affecting for instance 
the functions / and g, one cannot write the ^-MFG equations and one has to rely instead 
on the ^-Master equations (see jUj). 

To prove that these ty-Master equations are indeed more general than the ^-MFG equa- 
tions we enounce the following proposition: 

Proposition 4 (From ^-Master equations to Q-MFG equations). Let us consider a 
function {t,mi, ... ,m]\f) G [0,T] x i-^ {Ui(t,mi, . . . ,mN), . . . ,UN{t,mi, . . . ,m]\f)) ver- 
ifying the Q-Master equations. Let us then consider a function m : t G [0,T] i-^ 
(m(t, 1), . . . , m{t, N)) verifying: 

yi G TV, --m(t, i) = 
' d-" ^ ' ^ 

E ^^§^{mt,m{t,l),...,mit,N)) 

- E ^^§^{iUkit,mit,l),...,m{t,N)) 
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- Uj{t,m{t, 1), . . . ,m(t, A^)))fcgv(i)) m{t,j) 

- Ui{t, m{t, 1), . . . , m{t, N)))kevii)) m{t, i) 



with (m(0, 1), . . . , m(0, N)) = e Vn- 
If we define 

Mi G TV, n(t, i) = Ui{t, m{t, I),..., m{t, N)) 

then t £ [0, T] i— )■ {u{t, 1), . . . , u{t, N),m{t, 1), . . . , m{t, N)) is a solution of the Q-MFG equa- 
tions with the initial condition . 

Proof: 

Differentiating u{t,i) = Ui{t,m{t,l), . . . ,m{t,N)) with respect to t, one obtain tlie z*'* 
Hamilton- Jacobi equation tliat defines tlie Q-MFG equations. Tlie result is then straight- 
forward. □ 



3 Potential games and reduction to a planning prob- 
lem 

In this section, we provide a way to deduce the ^-Master equations that consist in N equa- 
tions from a single equation when the functions / and g verify the hypotheses of potential 
game^. We say that the above game is a potential game when there exist two functions 
F : (mi, . . . , miy) G J7 i-> F{mi, . . . , miy) and G : (mi, . . . , mAr) € i-^ G{mi, . . . ,mN) such 
that Vi G A/-, U = f{i, •) and |g = g{i, •) in n. 

To that purpose let us introduce a global optimization problem that we phrase a plan- 
ning problem^ This optimization problem consists in optimizing directly, with a macroscopic 
viewpoint, the different flows of agents (the preceding players being now controlled by a sin- 
gle agent, often referred to as a planner) between the nodes of the graph. 

We introduce for t S [0,r], m* S Vn and a given admissible control (function) X £ A, 
the payoff function 



T / N 

(F(m(s))-^L(z,(A,(.,j)), 



J{t,m\\) = yF{m{s))-}_^L{i,{Xs{tJ))^^^^,))m{s,i))ds + G{m{T)) 

where Vi G Af, m{t, i) = m\ and 

d 
ds 

The optimization problem we consider is, for a given G Vn'- 



Mi £ M,ys e [t,T],^m{s,i) = ^ Xsij,i)m{s,j) - ^ X^ii, j)m{s,i) 



sup .7(0, mO, A) 

Noteworthy, and contrary to the first optimization problem that was a stochastic control 
problem, this new problem is a deterministic control problem. 

We define the Hamilton-Jacobi equation associated to this deterministic control problem: 



^This is for instance the case if Vi G A/", f{i, •) and g{i, •) only depend on rrii. 

^The term ''^planning problem" must be understood here as the problem of a planner controlling the whole 
population of players. The planing problem studied in |1] is of a different nature. 
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Definition 4 (The t/-planning equation). The Q -planning equation consists in a single par- 
tial differential equation, the unknown being {t,m) & [Q,T] x Q i-^ ^{t,mi, . . . ,mjv).' 



5<I> 

— . . . ,mN) + y.{mi, . . . ,mAr, V$) + F{mi, . . .,mN) = 

with the terminal conditions ^{T, mi, . . . , tun) = G{mi, . . . , uin), where the hamiltonian is 
given by: 



N 

'H{mi,...,mN,p) = sup ^ 

i^i,j)ieM,jevii) i=i 



Our goal is to link this planning problem to the decentralized problem. The first step is 
to write the hamiltonian 7i in another way: 

Lemma 2. 

N 

nimi, . . .,mN,p) = ^TOii? (i, (Pj -Pi)j^y(^^^ 

Proof: 

Reordering the terms we can write 7i(mi, . . . , mN,p) as: 

sup J2 >'id(.Pj-Pi)-Hh{^i,j)jeV{i)) 
i=l iKj)jev(i) \jeV{i) 

Then using the definition of the hamiltonians in the decentralized problem, we get the re- 
sult. □ 



Proposition 5. Let us consider a function $ solution of the Q-planning equation. Then, 
^> restricted to [0,r] x Vn is the value function of the above planning problem, i.e.: 

V(t, m*) e [0, T] X Vn, Ht, m*) = sup J{t, m\ A) 



Moreover, if we define Vi G N, m{t, i) = ml and 

d 
ds 



yi eM,\/s e [t,T],^m{s,i) = Yj ^sU,i)m{s,j) - ^ \s{i,j)m{s, 



with 



= ^ ((1^^^' • • • ' - • • • ' J 

i/ten A is an optimal control for the planning problem. 
Proof- 
Let us consider an admissible control A G .4 and a repartition m* G Vn- 

Then, let us define: 

GAr,Vs G [t,r],^m*''"*'^(s,z) = Yl A,(i,z)m*'"**'^(s,i) - ^ A5(z,i)m*'"^''^(s,z) 

10 



with m{t, i) = m\. 
We can write 



9$ 



5] A,(j,i)m*'™''^(.,j)- 5] A,( 
.iev-i(i) iev(i) 

Hence, using the definition of <l>: 



4 = 1 

t-T 



with equahty when A is equal to A as defined in the Proposition, thanks to the preceding 
Lemma (this A being admissible because of our hypotheses on the hamiltonians, because $ 
is and because Vn is compact). 

As a consequence: 

J{t, m\ A) < m*) = J{t, m\ A) 
and this proves the result. □ 

Now, we are going to relate the function ^ to the ^-Master equations and prove that a 
solution to the planning problem can provide a solution to the decentralized problem, that 
is the initial mean field game. 

Proposition 6. Let us consider a functiot^ ^ solution of the Q-planning equation. 
Define Mi G TV, Vt € [0, T], Vm G Vn, Ui{t,mi, . . . ,171^) = ^^(t, mi, . . . , ttitv). Then, 
V$ = U = {Ui, . . . , U]sf) verifies the Q-Master equations. 
Consequently, if we define \/i G M, m(0, i) = for a given mP G Vn and 



Vi G 7V,Vs G [t,r], ^m(s,i) = ^ \s{j,i)m{s, j) - ^ \s{i, j)m{s,i) 



with 



^s{i,j) = ' ' — {s,m{s,l), . . . ,m{s, N)) - -—{s,m{s,l), . . . ,m{s, N)) 

then m is a Nash-MFG equilibrium and A is an optimal control for the initial mean field 
game (the decentralized problem). 

Proof: 

We need to prove the first assertion of the Proposition, then the other assertions are 
consequences of Proposition 3. 

Differentiating the ^/-planning equation with respect to nii, and using the above Lemma, 
we know since $ is that: 



^In reality, we do not need to be able to differentiate twice with respect to t 
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j=i kev{j) ^ * 

Now, using the fact that $ is C^, we have V(i, j) G A/"^, ^ = Hence: 

i=i fcGV(j) ^ " 3^ 

and this is exactly the z*'^ equation amongst the ^-Master equations, after reordering the 
terms. Hence, since the two terminal conditions are coherent, U = V$ is indeed solution of 
the ^-Master equations. □ 



Conclusion 

In this paper, we presented a way to deduce a solution to the mean field games equations in 
a graph from a smooth solution of a single equation Hamilton-Jacobi equation associated to 
a global deterministic control problem on the whole graph. Moreover, and as described on 
Figure 1 below, the Master equations - that "contain" MFG equations and have to be used 
in the case of games with common noise ~ can be reinterpreted as the equations verified by 
the partial differentials of solutions of this single Hamilton-Jacobi equation. 



Planning problem 


Dijferentiation 


Master equations 


Characteristics 


MFG equations 


1 HJ equation 




N equations 




2N equations 



This paper also opens the door to the important challenge of numerical resolution of 
Hamilton Jacobi equations in very high dimension either to solve the equations presented in 
this paper or as a numerical approximation to solutions of the Hamilton-Jacobi equations 
in infinite dimension presented in |14j . 
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